Processing of abdominal images

ABSTRACT

According to one embodiment there is provided a computer-automated image processing method applied to a four-dimensional (4D) image data set of a patient&#39;s abdomen, e.g. by dynamic contrast enhanced computer-assisted tomography (DCE-CT). One of the three-dimensional (3D) scan images is taken to as the reference volume and the others as target volumes. Before registration between the 3D scan images, the image data set is partitioned into an abdominal cavity domain, containing the organs inside the abdominal wall, and an abdominal wall domain including the abdominal wall and externally adjacent skeletal features, such as the spine and ribs. Registration is then carried out separately on the two domains to obtain two warp fields which are then merged into a 4D image data set of the whole volume for further use, which may be to carry out perfusion measurements, to display and to store the registered 4D image data set.

BACKGROUND OF THE INVENTION

Embodiments described herein relate generally to computer-automatedimage processing of time sequences of volume image data sets of theabdomen.

In the medical field, three-dimensional (3D) image data sets, i.e.volume image data sets, are collected by a variety oftechniques—referred to as modalities in the field—includingcomputer-assisted tomography (CT), magnetic resonance (MR), ultrasoundand positron-emission-tomography (PET).

For a number of years, perfusion in abdominal organs has been measuredusing so-called four-dimensional (4D) dynamic contrast enhanced CT(DCE-CT). In outline, the following procedure is applied. A bolus ofcontrast agent is injected. Then typically between 10 and 25 CT scansare acquired at intervals of a few seconds. The relevant vasculature andorgans are identified in each scan and their CT densities measured.Using a calibration curve, CT densities can be converted to contrastagent concentrations. The sequence of concentrations at each locus isplotted against time, providing a perfusion curve for the locus.

FIG. 1 shows an example perfusion curve intensity in Houndsfield Units(HU), which is proportional to perfusion, against time (T). Each pointon the curve is from the same locus in the patient as indicated by thearrows leading from the perspective 2D image panels.

Early work was based on time sequences of 2D slices, while more recentlythe advent of multi-slice scanners has enabled the rapid capture of timesequences of 3D CT volume images. The recent introduction of 320-sliceCT scanners that can acquire high-resolution CT images with up to 16 cmaxial extent in a single gantry revolution has made it feasible tocapture multiple sequential images of entire organs such as the kidneysor the liver while delivering a relatively small total X-ray dose.

If the patient is completely immobile both externally and internallythroughout the procedure, then the loci of interest need only beidentified in a single scan, and can be automatically transferred to allother scans. Clearly, this greatly reduces the interaction time neededto generate the perfusion results. In reality, respiration-inducedinternal motion provides a significant challenge to the imageprocessing. Two motion-reducing protocols are known. In one, the patientis instructed to hold their breath throughout the capture sequence.However, noting that patients are often sick, elderly, or both, asignificant number fail to achieve this. Instead, at some point in thecapture sequence they break the breath-hold with a deep gasping breaththat results in major motion (often more than 30 mm) of the abdominalorgans. An alternative protocol is to ask the patient to breathregularly but as shallowly or quietly as possible. In this case, themaximum extent of abdominal organ motion is reduced, at the expense ofubiquitous albeit smaller amounts of motion. Finally, not all patientseven succeed in lying still on the CT table so that whole-body motions,usually in effect small rotations about the patient's longitudinal axis,are not unknown.

One rather unsatisfactory approach is simply to identify and ignorethose volume images in which large organ motion is present. Automaticvolume registration provides a better approach. Automatic registrationhas been widely used in DCE-CT and in the related field of DCE-MRI foralmost two decades and is now available from a number of manufacturersas part of their perfusion analysis capabilities.

Respiration-induced motion affects the abdomen in a rather complexfashion. Put simply, the internal organs (liver, kidneys, spleen,pancreas, etc.) tend to move approximately axially (Z), driven by thedownward motion of the diaphragm. Deep breathing can cause overall axial(Z) displacements of 30 mm or more, with smaller motion in the coronal(Y) direction and, usually, least motion in the sagittal (X) axis. Ingeneral the different organs will move by different amounts. For apatient in the usual prone position, the vertebrae and rear abdominalwall remain approximately motionless. The front abdominal wall on theother hand tends to move mostly in the sagittal direction. The axialmotion of the internal organs and the relatively immobile spine andabdominal wall have a boundary that is effectively discontinuous inplaces; the organs appear to “slide” along the inner surface of theabdominal wall. Finally, the organs are not themselves rigid but cansubtly change in shape in addition to their overall gross movement.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is now described by way of example only with reference tothe following drawings.

FIG. 1 is a graph of CT density in Houndsfield Units (HU) at a specificlocus plotted against time during a 4D DCE-CT scan—a so-called perfusioncurve.

FIG. 2 shows a generic computer-assisted tomography (CT) scanner forgenerating volume data.

FIG. 3 schematically shows a computer for processing image data.

FIG. 4 schematically shows some of the features of the computer of FIG.3 in more detail.

FIG. 5 is a flow diagram showing process steps of an image processingmethod.

FIG. 6 is a 2D section volume rendered from a 3D CT scan of an abdomen.

FIG. 7 shows a corresponding 2D section to FIG. 6 obtained from the samevolume data set as FIG. 6 after the volume data set has been partitionedabout the abdomen wall.

FIG. 8 shows an example computer network.

DETAILED DESCRIPTION

FIG. 2 is a schematic perspective view of a generic scanner 2, mostespecially a computer-assisted tomography (CT) scanner, for obtaining a3D X-ray scan of a region of a patient 4. A patient's abdomen includingone or more organs or other anatomical features of interest is placedwithin a circular opening 6 of the scanner 2. A series of image slicesthrough the patient's abdomen is taken. Raw image data are derived fromthe scanner and could comprise a collection of one thousand 2D 512×512data subsets, for example. These data subsets, each representing a sliceof the region of the patient being studied, are combined to producevolume data. The volume data, which makes up a 3D image data set,comprise a collection of voxels each of which corresponds to a pixel inone of the slices. Thus the volume data are a 3D representation of thefeature imaged and various user-selected 2D projections (output images)of the 3D representation can be displayed (typically on a computermonitor).

Different imaging modalities (e.g. CT, MR, PET, ultrasound) typicallyprovide different image resolutions (i.e, voxel size), and the overallsize of the volume imaged will further depend on the nature of thestudy. However, in the following description, by way of concrete exampleit will be assumed that the volume data comprise an array of 512×512×32016-bit voxels arranged on a regular Cartesian grid defined by x-, y- andz-axes, with the voxels being spaced by 0.5 mm along each axis. Thiscorresponds to an overall imaged volume of around 25 cm×25 cm×16 cm,which is more than adequate to encompass an abdominal organ of interest,such as a kidney, the liver, the bowel, the spleen or the pancreas. Asis conventional, the volume data are aligned with transverse, sagittaland coronal planes. The xy-axes are in a transverse plane, the xz-axesare in a coronal plane and the yz-axes are in a sagittal plane.

FIG. 3 schematically illustrates a general purpose computer system 22configured to perform processing of volume data to generate twodimensional images. The computer 22 includes a central processing unit(CPU) 24, a read only memory (ROM) 26, a random access memory (RAM) 28,a hard disk drive 30, a display driver 32 and display 34 and a userinput/output (IO) circuit 36 with a keyboard 38 and mouse 40. Thesedevices are connected via a common bus 42. The computer 22 also includesa graphics card 44 connected via the common bus 42. In this example, thegraphics card is a Radeon X800XT visual processing unit manufactured byATI Technologies Inc., Ontario Canada. The graphics card includes agraphics processing unit (GPU) and random access memory tightly coupledto the GPU (GPU memory) (not shown in FIG. 3).

The CPU 24 may execute program instructions stored within the ROM 26,the RAM 28 or the hard disk drive 30 to carry out processing of signalvalues associated with voxels of volume data that may be stored withinthe RAM 28 or the hard disk drive 30. The RAM 28 and hard disk drive 30are collectively referred to as the system memory. The GPU may alsoexecute program instructions to carry out processing of volume datapassed to it from the CPU.

FIG. 4 schematically shows some of the features of the computer systemshown in FIG. 3 in more detail. The RAM 28 and hard disk drive 30 areshown collectively as a system memory 46. Volume data 48 obtained fromthe scanner 2 shown in FIG. 2 is stored in the system memory as shownschematically in the figure.

To assist in showing the different data transfer routes between featuresof the computer system 22, the common bus 42 shown in FIG. 3 isschematically shown in FIG. 5 as a series of separate bus connections 42a-d. A first bus connection 42 a connects between the system memory 46and the CPU 24. A second bus connection 42 b connects between the CPU 24and the graphics card 44. A third bus connection 42 c connects betweenthe graphics card 44 and the display 34. A fourth bus connection 42 dconnects between the user I/O 36 and the CPU 24. The CPU includes a CPUcache 50. The graphics card 44 includes a GPU 54 and a GPU memory 56.The GPU 54 includes circuitry for providing an accelerated graphicsprocessing interface 60, a GPU cache I/O controller 62, a processingengine 64 and a display I/O controller 66. The processing engine 64 isdesigned for optimized execution of the types of program instructionstypically associated with processing 3D image data sets and carrying out3D rendering of such data sets.

The user defines the required parameters using the keyboard 38 and mouse40 in combination with a menu of options displayed on the display 34,for example using conventional techniques.

The example implementations of the invention described below relate toperfusion measurements on internal abdominal organs, such as the liver,either kidney, spleen or pancreas. The 3D image is obtained from a CTscan, i.e. 3D X-ray image. To provide contrast for the X-rays, the bloodis typically dyed with a contrast agent having higher stopping powerthan the surrounding tissue. The patient is typically injected with abolus of the contrast agent during an initial phase of the imagingprocess so that the earliest images taken are free of contrast agenteffects, which then appear during the course of the image sequence.Iodine is a common contrast agent used for this purpose.

The following description presupposes that multiple 3D CT image datasets have been taken at short intervals from a patient using a suitableCT scanner, and is stored in system memory. Such data sets are sometimesreferred to as 4D with the fourth dimension being time, or morespecifically 4D dynamic contrast-enhanced CT (DCE-CT). Typically with2011 technology, a few tens of CT scans, e.g. 10-30 scans, are acquiredat intervals of a few seconds. The relevant vasculature and organs areidentified in each scan and their CT densities measured. At least oneinitial scan in the sequence will be taken before injection of thecontrast agent, or at least before the contrast agent can be expected tohave reached the organ of interest, thereby to provide a suitablereference volume for registration without presence of the high stoppingpower contrast agent. In those scans where the contrast agent ispresent, using a calibration curve, CT densities in Houndsfield Units(HUs) can be converted to contrast agent concentrations. The sequence ofconcentrations at each locus is plotted against time then provides aperfusion curve for the locus.

The 4D DCE-CT data on which image processing methods as described hereinwere tested were collected as follows. The method was tested on 22datasets each comprising a time-sequence of low-dose CT scans. Thenumber of scans was in the range 10-30 with an average of 16 scans.

After bolus injection of the contrast agent, time sequences of512×512×320 volume images were captured using a 320-detector row CTscanner (Aquilion ONE, Toshiba Medical Systems, Tochigi, Japan). TheX-ray source setting was 80 kV, 210 mA, resulting in a relatively lowtotal overall X-ray dose. The time sequence was: 10 scans at 3sintervals followed by a pause of 10s, then 3 scans at 7s intervalsfollowed by a second pause of 10s, and then a 3 further scans again atan interval of 7s. The pixel spacing within slices varied somewhat fromcase to case, but was typically about 0.65 mm, while the slice spacingwas 0.5 mm. Following CT reconstruction, the valid subset of each volumeimage consisted of a central cylinder of height approximately 11 cm,together with a conical region at each end that tapers from the radiusof the cylinder to a point at the axial extremity of the volume, givinga total height of 16 cm, Pixels lying outside the valid subset wereassigned a “padding” value to distinguish them from those having a validCT value.

Images were produced using low X-ray doses and as a result were quitenoisy following reconstruction. Noise was estimated by the followingmethod. An image was first shifted by one pixel in the X direction, thensubtracted from the original image. Assuming additive “white” noise, thestandard deviation of the difference image divided by √2 is an estimateof the noise standard deviation. Noise standard deviation values in theorder of 70 Hounsfield units (HU) were observed in most cases. Forcomparison, the intensity difference between a kidney and its adjacentfatty background is typically around 130 HU in a non-contrast scan, i.e.the signal to noise ratio is very low, in the order of two to one.

The method ensured that “padding” pixels lying outside the valid regionof interest (ROI; the cylinder plus 2 cones, see above) had no impact onthe registration. A 3D “abdomen domain” that included all of the bodypresent in the scan but excluded surrounding air and the CT scannertable was found automatically by thresholding, choosing the largestconnected component, and simplifying by a combination of binary imagemathematical morphology operations. Pixels lying outside the abdomendomain were also excluded from consideration by the registration enginewhich is to be described in more detail further below.

In order to make perfusion measurements on internal abdominal organs(e.g. liver, kidney) it is necessary for an image processing method tobe applied in order to register the multiple low-dose CT scans capturedin a time sequence.

However, a straightforward approach of applying non-rigid registrationto the complete volumes—whole volume non-rigid registration—does notperform well in all circumstances.

Whole volume non-rigid registration performs well if the data have beencaptured using a breath-hold protocol. However, it is clear from theresults that some patients find it impossible to hold their breath forthe requisite length of time, and movement artifacts appear in theimages obtained by whole volume non-rigid registration. As analternative to the breath-hold protocol, some data sets have beencaptured using a shallow-breathing protocol. Here the movement is lesspronounced. However, movement artifacts are still seen in processedimages that have been registered with whole volume non-rigidregistration.

The main underlying issues are two-fold.

First, respiration causes the internal organs to move in thesuperior-inferior (axial) direction, by more than 30 mm in some cases,while the spine, rib-cage, pelvis and abdominal wall tend to move less,and predominantly in the anterior-posterior (coronal) direction.

Second, the requirement for the CT scans to be made with as low a doseas possible results in the scanned images having very high noise levels(noise standard deviations of more than 150 HU have been observed). Highlevels of image noise are known to degrade the performance of non-rigidregistration.

An image processing method that performs better that a standardwhole-volume registration approach is now described.

FIG. 5 shows the steps of the principal steps of the method which are asfollows:

-   S1) Load: A 4D image data set made up of a time sequence of 3D image    data sets is loaded onto a computer for processing, for example from    a remote central file store of patient images.-   S2) Partition: A pre-contrast 3D image data set is chosen to be the    registration reference 3D image data set. The reference 3D image    data set is processed by the computer to partition the abdomen into    two regions, the “abdominal cavity” and the “abdominal wall plus    adjacent skeletal features of the rib-cage, pelvis and spine”. This    is done fully-automatically by image analysis.-   S3) Register: Each of the remaining 3D image data sets in the    sequence is registered with the reference 3D image data set. The two    regions in the reference 3D image data set are registered separately    in sub-steps S3.1 and S3.2—in each case using rigid registration. In    one example, non-rigid registration is carried out after rigid    registration in this part of the process flow.-   S4) Merge/Join: The resulting registrations are combined. In another    example, if the non-rigid registration was not carried out as part    of the registration step S3, it is carried out now after combining    the registered sub-volumes.-   S5) Store/Display: The combined registered 4D image data set is    stored and/or displayed.

The automated partitioning method is now described in more detail withreference to an axial CT scan of the abdominal region.

-   -   1. Choose a volume that was obtained when the patient, or at        least the patient's abdomen, did not contain any contrast agent.        This is important because the skeleton is separated from the        rest of the abdomen by intensity thresholding and the contrast        agent by definition has high intensity which would interfere        with this separation.    -   2. Apply a threshold in the order of 150 HU to the entire volume        to segment the bones from almost all other tissue.    -   3. Apply several mathematical morphology operations to separate        the spine. These are essentially standard, and are described in        more detail below.    -   4. Using other mathematical morphology operations, find a first        shell-like region containing ribs and/or pelvic bones (depending        on the axial location of the scanned volume), as described in        more detail below.    -   5. Apply a second threshold in the order of −600 HU to the        entire volume to separate all tissue from the air background.    -   6. Find a second shell-like region by applying several        mathematical morphology operations, as described in more detail        below. The second shell-like region comprises the skin,        subcutaneous fat and other exterior tissue that we refer to in        this document as the “abdominal wall”.

A suitable sequence of mathematical morphology operations to separatethe spine is as follows:

-   -   Dilate the above-threshold region by a 2 mm spherical        structuring element, and fill any internal holes, to join the        vertebrae and fill any hollow bones.    -   Erode the resulting region by a 5 mm spherical structuring        element. This will tend to separate the ribs from the spine.    -   Find the biggest connected component in the result of the        previous step, and dilate it by a 5 mm spherical structuring        element. This results in a slightly smoothed version of the        spine.    -   Replace each 2D axial slice of the spine by its filled minimum        enclosing convex polygon. This results in a smoother envelope        for the spinal region. We refer to the result as the “spine        domain”.

Note that here and elsewhere in this document, “structuring element”,“dilate”, “erode”, “open”, “close” and “fill” are standard “mathematicalmorphology” terms and operations. “Domain” is a synonym of “mask”. Thetext book Serra J., Image Analysis and Mathematical Morphology, 1982,ISBN 0126372403 provides details of mathematical morphology operationsof the kind applied here, and is incorporated herein by reference in itsentirety.

The mathematical morphology operations used to find the first shell-likeregion (ribs and/or pelvic bones) can be implemented as follows:

-   -   Take the cleaned-up above-threshold region found in the first        dilation step mentioned above, and open using a 1 mm spherical        structuring element.    -   Subtract the spine domain found above. The result comprises the        ribs and/or pelvis.    -   Dilate the ribs/pelvis with a cylindrical structuring element of        radius 5 mm and length (in the axial direction) 40 mm. This        converts the ribs/pelvis to a shell-like “bone curtain domain”        that is “hanging” in the axial direction.    -   Take the union of the bone curtain domain and the spine domain.        This will usually comprise a single connected component that        represents the bone region that is to be included as part of the        abdominal wall rather than the abdominal cavity. This is        referred to in the following as the “bone domain”.

The mathematical morphology operations used to find the secondshell-like region (skin, subcutaneous fat and other exterior tissue) isas follows in one example:

-   -   Find the largest connect component of the above-threshold        region.    -   Replace each axial slice by the filled minimum enclosing convex        polygon. This fills air gaps in the bowel, the bottom of the        lungs, etc. and results in a “whole abdomen domain”.    -   Take the complement of the whole abdomen domain, resulting in        the region outside the body, including in particular the scanner        table as well as the surrounding air. Dilate this region by 25        mm, with the result that it extends 25 mm into the body. The        resulting domain now comprises the outside 25 mm shell of the        body plus all of the exterior background, and is known as the        “abdominal wall shell”. (It will be appreciated that the choice        of dimension may be varied as desired.)    -   Take the union of the bone domain and the abdominal wall shell,        resulting in the “abdominal wall plus adjacent skeletal features        of the rib-cage, pelvis and spine”. Subtract this from the        entire image volume. The result is the “abdominal cavity”        domain. The abdominal wall and abdominal cavity are now disjoint        and together comprise the entire volume.    -   Next the abdominal wall plus adjacent skeletal features, and the        abdominal cavity, are made separate from each other by opening        up a 15 mm wide space at their common boundary. In detail this        can be achieved as follow: The abdominal cavity domain is first        opened using a 15 mm spherical structuring element, thus        smoothing its boundary by removing any small intrusions into the        true abdominal wall. The final abdominal wall and bone domain is        then constructed by dilating the abdominal cavity by 15 mm, then        subtracting it from the whole abdomen domain found above.

The result is two domains, separated by a 15 mm-wide space, with smoothsurfaces. The first comprises almost all of the abdominal internalorgans (liver, kidney, spleen, pancreas, gall bladder, bowel); thesecond contains the spine, the ribs, the pelvis if present, everythingbetween the skeleton and the skin, and most of the remaining abdominalwall (e.g. the anterior wall, which contains no bones).

FIG. 6 is a 2D volume rendered sectional view of a patient's abdomentaken by a CT scan, with no contrast agent present.

FIG. 7 is a corresponding image after partition has separated out theabdominal wall plus adjacent skeletal features domain, thereby onlyshowing the abdominal cavity domain.

After completion of the partitioning, the subsequent steps are separateregistration within each of the two domains or sub-volumes followed byjoining or merging the resulting just-registered sub-volumes.

Registration is carried out to a reference volume separately to each ofthe two regions (abdominal cavity region and abdominal wall etc.region). Each of the two regions is represented by a “domain” or “mask”applied to the image volume chosen as the reference volume forregistration. Each other perfusion-phase image volume (referred tohereinafter as the “target” volume) is then registered to the referencevolume using only the portion of the reference volume contained withinone domain. The registration comprises an initial “rigid” registrationto take account of displacement, and optionally rotation to take accountof any slight rotation between the target and reference, followed bynon-rigid registration to accommodate soft-tissue distortions. Ingeneral, a registration process seeks to maximize a similarity measurebetween the reference image and the warped target or floating image. Thesimilarity measure may be, for example, mutual information (MI) ornormalized mutual information (NMI). A discussion of similarity measurescan be found elsewhere in the literature, for example in the articleCrum W. R., Hill D. L. G., Hawkes D. J. (2003) Information theoreticsimilarity measures in non-rigid registration. IPMI-2003, pp. 378-387which is incorporated herein by reference in its entirety. Theregistration code was optimized for rapid execution on a modernmulticore PC.

The result of the two separate registrations is two warp fields.Generally speaking, a warp field between two 3D data sets provides, forevery voxel coordinate of the reference image, a relative offset to acorresponding location in the target or floating image. A warp field inCartesian coordinates will thus comprise three orthogonal components,one for each of the 3 coordinate dimensions. The two warp fields in thiscase will be defined on two disjoint but closely adjacent domains: onefor the abdominal cavity, and one for the abdominal wall etc.

One could simply combine the two warp-fields into one single warp fieldby extracting values from each within its own domain of definition, butthis can potentially result in artifacts in the final combined image,such as gaps, tears, folds and occlusions.

Instead, it is better to erode the two domains until the subsets of thewarp-fields defined on the eroded domains are consistent (i.e. no gaps,tears, folds, or occlusions), and then propagate the two warp-fieldsinto the eroded regions in a consistent manner.

One specific way to implement this is a distance-transform-weightedpropagation of the two warp-fields into the gap between the two domains,this being the 15 mm gap referred to earlier.

Another specific way, which is probably superior, is to use the methoddescribed above to merge the warp-fields obtained on the two domainsfrom the rigid registration steps before applying non-rigidregistration, and then apply non-rigid registration, seeded by theresulting warp-field, to the entire volume. This approach where theregistration of the abdominal cavity is “blended” with the remainder ofthe abdomen can incorporate teachings from prior art on atlas-basedsegmentation in which individual organs or organ components are firstrigidly registered, then the registrations are combined in a consistentmanner to seed a non-rigid registration stage using approaches describedin the literature, for example in the following two articles thecontents of both of which are incorporated herein by reference in theirentirety:

-   -   Zhuang X., Hawkes D. J., Crum W. R., Boubertakh R., Uribe S.,        Atkinson D., Batchelor P., Schaeffter T., Razavi R.,        Hill D. L. G. (2008) Robust registration between cardiac MRI        images and atlas for segmentation propagation, SPIE Medical        Imaging, vol. 6914, pp. 7    -   Park H., Bland P. H., Meyer C. R., Construction of an Abdominal        Probabilistic Atlas and its Application in Segmentation, IEEE        TRANSACTIONS ON MEDICAL IMAGING, VOL 22, NO. 4, APRIL 2003,        pages 483-492

The person skilled in the art is familiar with techniques for mergingwarp fields so could use either of the above-mentioned ways or someother alternative way to achieve the merge.

The method has been fully implemented and tested on the 22 datasetsreferred to further above, each comprising an average of about 18consecutive CT scans of the abdomen. A visual comparison shows that themethod deals better with large motions of the internal organs than awhole-volume registration method, while being more or lessindistinguishable from a whole-volume registration method when motion issmall.

Methods described herein can be used within a hospital environment. Inthis case, the methods may usefully be integrated into a stand-alonesoftware application, or with a Picture Archiving and CommunicationSystem (PACS). A PACS is a hospital-based computerized network which canstore volume data representing diagnostic images of different types in adigital format organized in a single central archive. For example,images may be stored in the Digital Imaging and Communications inMedicine (DICOM) format. Each image has associated patient informationsuch as the name and date of birth of the patient also stored in thearchive. The archive is connected to a computer network provided with anumber of workstations, so that users all around the hospital site canaccess and process patient data as needed. Additionally, users remotefrom the site may be permitted to access the archive over the Internet.

In the context of the present invention, therefore, a plurality of imagevolume data sets can be stored in a PACS archive, and acomputer-implemented method of generating movies or other output imagesof a chosen one of the volume data sets according to embodiments of theinvention can be provided on a workstation connected to the archive viaa computer network. A user such as a radiologist, a consultant, or aresearcher can thereby access any volume data set from the workstation,and generate and display movies or other images, such as a stills imageof a heart feature at a particular phase of interest, using methodsembodying the invention.

FIG. 8 shows an example computer network. The network 150 comprises alocal area network in a hospital 152. The hospital 152 is equipped witha number of workstations 154 which each have access, via the local areanetwork, to a hospital computer server 156 having an associated storagedevice 158. A PACS archive is stored on the storage device 158 so thatdata in the archive can be accessed from any of the workstations 154.One or more of the workstations 154 has access to a graphics card and tosoftware for computer-implementation of methods described herein. Thesoftware may be stored locally at the or each workstation 154, or may bestored remotely and downloaded over the network 150 to a workstation 154when needed. In other example, methods described herein may be executedon the computer server with the workstations 154 operating as terminals.For example, the workstations may be configured to receive user inputdefining a desired image data set and to display resulting images whilevolume rendering is performed elsewhere in the system. Also, a number ofmedical imaging devices 160, 162, 164, 166 are connected to the hospitalcomputer server 156.

Image data collected with the devices 160, 162, 164, 166 can be storeddirectly into the PACS archive on the storage device 156. Thus patientimages can be rendered and viewed immediately after the correspondingvolume data are recorded, so that a swift diagnosis can be obtained inthe event of medical emergency. The local area network is connected tothe Internet 168 by a hospital Internet server 170, which allows remoteaccess to the PACS archive. This is of use for remote accessing of thedata and for transferring data between hospitals, for example, if apatient is moved, or to allow external research to be undertaken.

In the described embodiments, a computer implementation employingcomputer program code for storage on a data carrier or in memory can beused to control the operation of the CPU and GPU of the computer system.The computer program can be supplied on a suitable carrier medium, forexample a storage medium such as solid state memory, magnetic, opticalor magneto-optical disk or tape based media. Alternatively, it can besupplied on a transmission medium, for example a medium with a carriersuch as a telephone, radio or optical channel.

A computer program product bearing machine readable instructions forcarrying out the method is disclosed.

A computer loaded with and operable to execute machine readableinstructions for carrying out the method is disclosed.

A computer program product is disclosed. Examples of a computer programproduct bearing machine readable instructions for carrying out themethod described above are the mass storage device HDD 30 of FIG. 3, theROM 26 of FIG. 3, the RAM 28 of FIG. 3 and the system memory 46 of FIG.4, and the servers 156 or 170 of FIG. 8. Other forms of computer programproduct include a spinning disk based storage device such as a CD orDVD, or a USB flash memory device.

Examples of a computer loaded with and operable to execute machinereadable instructions for carrying out the method described above arethe computer of FIG. 3, the computer of FIG. 4, and individual elements,e.g. terminals 154 or collective multiple elements of the computernetwork system shown in FIG. 8, e.g. server 156 or 170 in combinationwith one or more of the terminals 154 or computers provided with themedical imaging devices 160, 162, 164 or 166.

Examples of a computer program product bearing machine readableinstructions for carrying out the method described above are the massstorage device HDD 30 of FIG. 3, the ROM 26 of FIG. 3, the RAM 28 ofFIG. 3 and the system memory 46 of FIG. 4, and the servers 156 or 170 ofFIG. 8. Other forms of computer program product include a spinning diskbased storage device such as a CD or DVD, or a USB flash memory device.

Embodiments of the invention may include incorporating the methods andassociated computer programs described herein as a component in a volumerendering application.

While the method has been described with reference to 3D image data setscollected by computer-assisted tomography (CT) scanners, it is moregenerally applicable to imaging of other 3D and indeed 4D or higherdimensionality data sets obtained from a wide variety of image capturedevices and a wide variety of objects.

For example, the method may be applied to a variety of imaging typesused in the medical field, referred to as modalities. In particular, themethods described herein may be applied to 3D image data sets collectedby computer-assisted tomography (CT) scanners, particularly theabove-described DCE-CT, magnetic resonance (MR) scanners, particularlyDCE-MRI, ultrasound scanners and positron-emission-tomography (PET)systems.

These 3D data sets are sometimes referred to as volume data. In medicalimaging, 3D image data sets are generally large. Sizes of between 0.5Gigabytes and 8 Gigabytes are not uncommon. For example, a medical imagedata set might comprise 1024×1024×320 16-bit voxels which corresponds toapproximately 1 Gigabytes of data. From this an image comprising1024×1024 16-bit pixels might be rendered.

In summary, the method thus involves separation of the abdominal cavityfrom the abdominal wall followed by separate registration of the tworesulting sub-volumes.

More particularly a computer-automated image processing method has beendescribed, comprising: (a) accessing (e.g. from a memory device such asthe GPU memory 56, the system memory 46, the HDD 30, the hospitalcomputer server 156 or its associated storage device 158) a 4D imagedata set comprising a plurality of 3D image data sets of a volume takenin time sequence, wherein the volume includes at least part of theabdomen of a living subject; (b) selecting (e.g. with instructionsrunning on a processor such as the CPU 24 or the GPU 54) one of the 3Dimages to serve as a reference volume 3D image data set; (c)partitioning (e.g. with instructions running on a processor such as theCPU 24 or the GPU 54) the reference volume 3D image data set into anabdominal cavity domain inside the abdominal wall and an abdominal walldomain including the abdominal wall and externally adjacent anatomicalfeatures; (d) registering (e.g. with instructions running on a processorsuch as the CPU 24 or the GPU 54) each of the 3D image data sets to thereference volume 3D image data set for the abdominal cavity domain toobtain a first warp field defined over the abdominal cavity domain; (e)registering (e.g. with instructions running on a processor such as theCPU 24 or the GPU 54) each of the 3D image data sets to the referencevolume 3D image data set for the abdominal wall domain to obtain asecond warp field defined over the abdominal wall domain; (f) for eachof the 3D image data sets, merging (e.g. with instructions running on aprocessor such as the CPU 24 or the GPU 54) the first and second warpfields into a single consistent merged warp field defined over the wholevolume; and (g) using each of the merged warp fields to bring thecorresponding 3D image data set into registration with the referencevolume 3D image data set (e.g. with instructions running on a processorsuch as the CPU 24 or the GPU 54).

It has been further described how the first and second warp fields maybe merged into the merged warp field by eroding the abdominal cavitydomain with the abdominal wall domain until adjacent portions of thefirst and second warp fields are consistent, and then propagating thefirst and second warp fields into the eroded regions.

In some embodiments, the 4D image data set represents a study in which acontrast agent has been introduced to the subject, this being manifestedby some, but not all, of the 3D image data sets having voxels affectedby the contrast agent, and wherein the reference volume 3D image dataset is selected to be a 3D image data set whose voxels are free ofeffects of the contrast agent. In particular, the study can be aperfusion study of an abdominal organ contained in the abdominal cavitydomain.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods, computers andcomputer program products described herein may be embodied in a varietyof other forms; furthermore, various omissions, substitutions andchanges in the form of the methods and systems described herein may bemade without departing from the spirit of the inventions. Theaccompanying claims and their equivalents are intended to cover suchforms or modifications as would fall within the scope and spirit of theinventions.

1. A computer-automated image processing method, comprising: accessing a4D image data set comprising a plurality of 3D image data sets of avolume taken in time sequence, wherein the volume includes at least partof the abdomen of a living subject; selecting one of the 3D images toserve as a reference volume 3D image data set; partitioning thereference volume 3D image data set into an abdominal cavity domaininside the abdominal wall and an abdominal wall domain including theabdominal wall and externally adjacent anatomical features; registeringeach of the 3D image data sets to the reference volume 3D image data setfor the abdominal cavity domain to obtain a first warp field definedover the abdominal cavity domain; registering each of the 3D image datasets to the reference volume 3D image data set for the abdominal walldomain to obtain a second warp field defined over the abdominal walldomain; for each of the 3D image data sets, merging the first and secondwarp fields into a single consistent merged warp field defined over thewhole volume; using each of the merged warp fields to bring thecorresponding 3D image data set into registration with the referencevolume 3D image data set.
 2. The method of claim 1, wherein each of theregistrations comprises a rigid registration
 3. The method of claim 2,wherein each of the registrations comprises a non-rigid registration. 4.The method of claim 2, wherein a non-rigid registration is applied tothe merged warp fields.
 5. The method of claim 1, wherein the 4D imagedata set represents a study in which a contrast agent has beenintroduced to the subject, this being manifested by some, but not all,of the 3D image data sets having voxels affected by the contrast agent,and wherein the reference volume 3D image data set is selected to be a3D image data set whose voxels are free of effects of the contrastagent.
 6. The method of claim 5, wherein the study is a perfusion studyof an abdominal organ contained in the abdominal cavity domain.
 7. Themethod of claim 1, further comprising: volume rendering the processed 4Dimage data set into a sequence of 2D image data sets; and displaying the2D image data sets in sequence to a user.
 8. A computer program productbearing machine readable instructions for carrying out the method ofclaim
 1. 9. A computer loaded with and operable to execute machinereadable instructions for carrying out the method of claim 1.